{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "yeadDkMiISin"
      },
      "source": [
        "# Gemini API: Function calling with Python"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "lEXQ3OwKIa-O"
      },
      "source": [
        "<table align=\"left\">\n",
        "  <td>\n",
        "    <a target=\"_blank\" href=\"https://colab.research.google.com/github/google-gemini/cookbook/blob/main/quickstarts/Function_calling.ipynb\"><img src=\"https://www.tensorflow.org/images/colab_logo_32px.png\" />Run in Google Colab</a>\n",
        "  </td>\n",
        "</table>\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "df1767a3d1cc"
      },
      "source": [
        "Function calling lets developers create a description of a function in their code, then pass that description to a language model in a request. The response from the model includes the name of a function that matches the description and the arguments to call it with. Function calling lets you use functions as tools in generative AI applications, and you can define more than one function within a single request.\n",
        "\n",
        "This notebook provides code examples to help you get started."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "9OEoeosRTv-5"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m137.4/137.4 kB\u001b[0m \u001b[31m933.2 kB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25h"
          ]
        }
      ],
      "source": [
        "!pip install -U -q google-generativeai # Install the Python SDK"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "TS9l5igubpHO"
      },
      "outputs": [],
      "source": [
        "import google.generativeai as genai"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "x-hHZfLZ7FfH"
      },
      "source": [
        "## Set up your API key\n",
        "\n",
        "To run the following cell, your API key must be stored it in a Colab Secret named `GOOGLE_API_KEY`. If you don't already have an API key, or you're not sure how to create a Colab Secret, see the [Authentication](https://github.com/google-gemini/cookbook/blob/main/quickstarts/Authentication.ipynb) quickstart for an example."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "ab9ASynfcIZn"
      },
      "outputs": [],
      "source": [
        "from google.colab import userdata\n",
        "GOOGLE_API_KEY=userdata.get('GOOGLE_API_KEY')\n",
        "genai.configure(api_key=GOOGLE_API_KEY)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "3f383614ec30"
      },
      "source": [
        "## Function calling basics"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "b82c1aecb657"
      },
      "source": [
        "To use function calling, pass a list of functions to the `tools` parameter when creating a [`GenerativeModel`](https://ai.google.dev/api/python/google/generativeai/GenerativeModel). The model uses the function name, docstring, parameters, and parameter type annotations to decide if it needs the function to best answer a prompt.\n",
        "\n",
        "> Important: The SDK converts function parameter type annotations to a format the API understands (`glm.FunctionDeclaration`). The API only supports a limited selection of parameter types, and the Python SDK's automatic conversion only supports a subset of that: `AllowedTypes = int | float | bool | str | list['AllowedTypes'] | dict`"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "42b27b02d2f5"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "genai.GenerativeModel(\n",
              "    model_name='models/gemini-1.0-pro',\n",
              "    generation_config={},\n",
              "    safety_settings={},\n",
              "    tools=<google.generativeai.types.content_types.FunctionLibrary object at 0x7a99071b61d0>,\n",
              ")"
            ]
          },
          "execution_count": 5,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "def add(a:float, b:float):\n",
        "    \"\"\"returns a + b.\"\"\"\n",
        "    return a+b\n",
        "\n",
        "def subtract(a:float, b:float):\n",
        "    \"\"\"returns a - b.\"\"\"\n",
        "    return a-b\n",
        "\n",
        "def multiply(a:float, b:float):\n",
        "    \"\"\"returns a * b.\"\"\"\n",
        "    return a*b\n",
        "\n",
        "def divide(a:float, b:float):\n",
        "    \"\"\"returns a / b.\"\"\"\n",
        "    return a*b\n",
        "\n",
        "model = genai.GenerativeModel(model_name='gemini-1.0-pro',\n",
        "                              tools=[add, subtract, multiply, divide])\n",
        "\n",
        "model"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "UzUgtaY99BTg"
      },
      "source": [
        "## Automatic function calling"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "d5fd91032a1e"
      },
      "source": [
        "Function calls naturally fit in to [multi-turn chats](https://ai.google.dev/api/python/google/generativeai/GenerativeModel#multi-turn) as they capture a back and forth interaction between the user and model. The Python SDK's [`ChatSession`](https://ai.google.dev/api/python/google/generativeai/ChatSession) is a great interface for chats because handles the conversation history for you, and using the parameter `enable_automatic_function_calling` simplifies function calling even further:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "d3b91c855257"
      },
      "outputs": [],
      "source": [
        "chat = model.start_chat(enable_automatic_function_calling=True)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "1481a6159399"
      },
      "source": [
        "With automatic function calling enabled, `ChatSession.send_message` automatically calls your function if the model asks it to.\n",
        "\n",
        "In the following example, the result appears to simply be a text response containing the correct answer:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "81d8def3d865"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "'There are 2508 mittens in total.'"
            ]
          },
          "execution_count": 7,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "response = chat.send_message('I have 57 cats, each owns 44 mittens, how many mittens is that in total?')\n",
        "response.text"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "951c0f83f72e"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "2508"
            ]
          },
          "execution_count": 8,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "57*44"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "7731e35f2383"
      },
      "source": [
        "However, by examining the chat history, you can see the flow of the conversation and how function calls are integrated within it.\n",
        "\n",
        "The `ChatSession.history` property stores a chronological record of the conversation between the user and the Gemini model. Each turn in the conversation is represented by a [`glm.Content`](https://ai.google.dev/api/python/google/ai/generativelanguage/Content) object, which contains the following information:\n",
        "\n",
        "*   **Role**: Identifies whether the content originated from the \"user\" or the \"model\".\n",
        "*   **Parts**: A list of [`glm.Part`](https://ai.google.dev/api/python/google/ai/generativelanguage/Part) objects that represent individual components of the message. With a text-only model, these parts can be:\n",
        "    *   **Text**: Plain text messages.\n",
        "    *   **Function Call** ([`glm.FunctionCall`](https://ai.google.dev/api/python/google/ai/generativelanguage/FunctionCall)): A request from the model to execute a specific function with provided arguments.\n",
        "    *   **Function Response** ([`glm.FunctionResponse`](https://ai.google.dev/api/python/google/ai/generativelanguage/FunctionResponse)): The result returned by the user after executing the requested function.\n",
        "\n",
        " In the previous example with the mittens calculation, the history shows the following sequence:\n",
        "\n",
        "1.  **User**: Asks the question about the total number of mittens.\n",
        "1.  **Model**: Determines that the multiply function is helpful and sends a FunctionCall request to the user.\n",
        "1.  **User**: The `ChatSession` automatically executes the function (due to `enable_automatic_function_calling` being set) and sends back a `FunctionResponse` with the calculated result.\n",
        "1.  **Model**: Uses the function's output to formulate the final answer and presents it as a text response."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "9f7eff1e8e60"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "user -> [{'text': 'I have 57 cats, each owns 44 mittens, how many mittens is that in total?'}]\n",
            "--------------------------------------------------------------------------------\n",
            "model -> [{'function_call': {'name': 'multiply', 'args': {'a': 57.0, 'b': 44.0}}}]\n",
            "--------------------------------------------------------------------------------\n",
            "user -> [{'function_response': {'name': 'multiply', 'response': {'result': 2508.0}}}]\n",
            "--------------------------------------------------------------------------------\n",
            "model -> [{'text': 'There are 2508 mittens in total.'}]\n",
            "--------------------------------------------------------------------------------\n"
          ]
        }
      ],
      "source": [
        "for content in chat.history:\n",
        "    print(content.role, \"->\", [type(part).to_dict(part) for part in content.parts])\n",
        "    print('-'*80)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "2471fd72f05e"
      },
      "source": [
        "In general the state diagram is:\n",
        "\n",
        "<img src=\"https://ai.google.dev/images/tutorials/function_call_state_diagram.png\" alt=\"The model can always reply with text, or a FunctionCall. Iff the model sends a FunctionCall the user must reply with a FunctionResponse\" width=50%>\n",
        "\n",
        "The model can respond with multiple function calls before returning a text response, and function calls come before the text response."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "eea8e3a0b89f"
      },
      "source": [
        "## Manual function calling"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "9610f3465a69"
      },
      "source": [
        "For more control, you can process [`glm.FunctionCall`](https://ai.google.dev/api/python/google/ai/generativelanguage/FunctionCall) requests from the model yourself. This would be the case if:\n",
        "\n",
        "- You use a `ChatSession` with the default `enable_automatic_function_calling=False`.\n",
        "- You use `GenerativeModel.generate_content` (and manage the chat history yourself)."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "34ffab0bf365"
      },
      "source": [
        "The following example is a rough equivalent of the [function calling single-turn curl sample](https://ai.google.dev/docs/function_calling#function-calling-single-turn-curl-sample) in Python. It uses functions that return (mock) movie playtime information, possibly from a hypothetical API:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "46ba0fa3d09a"
      },
      "outputs": [],
      "source": [
        "def find_movies(description: str, location: str = ''):\n",
        "  \"\"\"find movie titles currently playing in theaters based on any description, genre, title words, etc.\n",
        "\n",
        "  Args:\n",
        "      description: Any kind of description including category or genre, title words, attributes, etc.\n",
        "      location: The city and state, e.g. San Francisco, CA or a zip code e.g. 95616\n",
        "  \"\"\"\n",
        "  return ['Barbie', 'Oppenheimer']\n",
        "\n",
        "def find_theaters(location: str, movie: str = ''):\n",
        "    \"\"\"Find theaters based on location and optionally movie title which are is currently playing in theaters.\n",
        "\n",
        "    Args:\n",
        "        location: The city and state, e.g. San Francisco, CA or a zip code e.g. 95616\n",
        "        movie: Any movie title\n",
        "    \"\"\"\n",
        "    return ['Googleplex 16', 'Android Theatre']\n",
        "\n",
        "def get_showtimes(location:str, movie:str, theater:str, date:str):\n",
        "    \"\"\"\n",
        "    Find the start times for movies playing in a specific theater.\n",
        "\n",
        "    Args:\n",
        "      location: The city and state, e.g. San Francisco, CA or a zip code e.g. 95616\n",
        "      movie: Any movie title\n",
        "      thearer: Name of the theater\n",
        "      date: Date for requested showtime\n",
        "    \"\"\"\n",
        "    return ['10:00', '11:00']"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Ck-hdu5N8VlR"
      },
      "source": [
        "Use a dictionary to make looking up functions by name easier later on. You can also use it to pass the array of functions to the `tools` parameter of `GenerativeModel`."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "8i3SKdy18WHu"
      },
      "outputs": [],
      "source": [
        "functions = {\n",
        "    'find_movies': find_movies,\n",
        "    'find_theaters': find_theaters,\n",
        "    'get_showtimes': get_showtimes,\n",
        "}\n",
        "\n",
        "model = genai.GenerativeModel(model_name='gemini-1.0-pro',\n",
        "                              tools=functions.values())"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "11631c6e2b10"
      },
      "source": [
        "After using `generate_content()` to ask a question, the model requests a `function_call`:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "5e3b9c84d883"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "[function_call {\n",
              "  name: \"find_theaters\"\n",
              "  args {\n",
              "    fields {\n",
              "      key: \"location\"\n",
              "      value {\n",
              "        string_value: \"Mountain View, CA\"\n",
              "      }\n",
              "    }\n",
              "    fields {\n",
              "      key: \"movie\"\n",
              "      value {\n",
              "        string_value: \"Barbie\"\n",
              "      }\n",
              "    }\n",
              "  }\n",
              "}\n",
              "]"
            ]
          },
          "execution_count": 25,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "response = model.generate_content('Which theaters in Mountain View show the Barbie movie?')\n",
        "response.candidates[0].content.parts"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "kuldoypuAC1i"
      },
      "source": [
        "Since this is not using a `ChatSession` with automatic function calling, you have to call the function yourself.\n",
        "\n",
        "A very simple way to do this would be with `if` statements:\n",
        "\n",
        "```python\n",
        "if function_call.name == 'find_theaters':\n",
        "  find_theaters(**function_call.args)\n",
        "elif ...\n",
        "```\n",
        "\n",
        "However, since we made the `functions` dictionary earlier, we can simplify this to:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "rjkZ8MA00Coc"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "['Googleplex 16', 'Android Theatre']\n"
          ]
        }
      ],
      "source": [
        "def call_function(function_call, functions):\n",
        "  function_name = function_call.name\n",
        "  function_args = function_call.args\n",
        "  return functions[function_name](**function_args)\n",
        "\n",
        "part = response.candidates[0].content.parts[0]\n",
        "\n",
        "# Check if it's a function call; in real use you'd need to also handle text\n",
        "# responses as you won't know what the model will respond with.\n",
        "if part.function_call:\n",
        "  result = call_function(part.function_call, functions)\n",
        "\n",
        "print(result)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "XLWrHOatBtRz"
      },
      "source": [
        "Finally, pass the response plus the message history to the next `generate_content()` call to get a final text response from the model."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "gdb62GstAD_3"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Theaters showing Barbie in Mountain View, CA are: Googleplex 16, Android Theatre\n"
          ]
        }
      ],
      "source": [
        "import google.ai.generativelanguage as glm\n",
        "from google.protobuf.struct_pb2 import Struct\n",
        "\n",
        "# Put the result in a protobuf Struct\n",
        "s = Struct()\n",
        "s.update({'result': result})\n",
        "\n",
        "# Update this after https://github.com/google/generative-ai-python/issues/243\n",
        "function_response = glm.Part(\n",
        "    function_response=glm.FunctionResponse(name='find_theaters', response=s))\n",
        "\n",
        "# Build the message history\n",
        "messages = [\n",
        "    {'role':'user',\n",
        "     'parts': ['Which theaters in Mountain View show the Barbie movie?.']},\n",
        "    {'role':'model',\n",
        "     'parts': response.candidates[0].content.parts},\n",
        "    {'role':'user',\n",
        "     'parts': [function_response]}\n",
        "]\n",
        "\n",
        "# Generate the next response\n",
        "response = model.generate_content(messages)\n",
        "print(response.text)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "e0a3173919ca"
      },
      "source": [
        "## Next Steps"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "7c2f31504490"
      },
      "source": [
        "Useful API references:\n",
        "\n",
        "- The [genai.GenerativeModel](https://ai.google.dev/api/python/google/generativeai/GenerativeModel) class\n",
        "  - Its [GenerativeModel.generate_content](https://ai.google.dev/api/python/google/generativeai/GenerativeModel#generate_content) method builds a [glm.GenerateContentRequest](https://ai.google.dev/api/python/google/ai/generativelanguage/GenerateContentRequest) behind the scenes.\n",
        "    - The request's `.tools` field contains a list of 1 [glm.Tool](https://ai.google.dev/api/python/google/ai/generativelanguage/Tool) object.\n",
        "    - The tool's `function_declarations` attribute contains a list of [FunctionDeclarations](https://ai.google.dev/api/python/google/ai/generativelanguage/FunctionDeclaration) objects.\n",
        "- The [response](https://ai.google.dev/api/python/google/ai/generativelanguage/GenerateContentResponse) may contain a [glm.FunctionCall](https://ai.google.dev/api/python/google/ai/generativelanguage/FunctionCall), in `response.candidates[0].contents.parts[0]`.\n",
        "- if `enable_automatic_function_calling` is set the [genai.ChatSession](https://ai.google.dev/api/python/google/generativeai/ChatSession) executes the call, and sends back the [glm.FunctionResponse](https://ai.google.dev/api/python/google/ai/generativelanguage/FunctionResponse).\n",
        "- In response to a [FunctionCall](https://ai.google.dev/api/python/google/ai/generativelanguage/FunctionCall) the model always expects a [FunctionResponse](https://ai.google.dev/api/python/google/ai/generativelanguage/FunctionResponse).\n",
        "- If you reply manually using [chat.send_message](https://ai.google.dev/api/python/google/generativeai/ChatSession#send_message) or [model.generate_content](https://ai.google.dev/api/python/google/generativeai/GenerativeModel#generate_content) remember thart the API is stateless you have to send the whole conversation history (a list of [content](https://ai.google.dev/api/python/google/ai/generativelanguage/Content) objects), not just the last one containing the `FunctionResponse`."
      ]
    }
  ],
  "metadata": {
    "colab": {
      "name": "Function_calling.ipynb",
      "toc_visible": true
    },
    "google": {
      "image_path": "/site-assets/images/share.png",
      "keywords": [
        "examples",
        "googleai",
        "samplecode",
        "python",
        "embed",
        "function"
      ]
    },
    "kernelspec": {
      "display_name": "Python 3",
      "name": "python3"
    }
  },
  "nbformat": 4,
  "nbformat_minor": 0
}
